cpi uiaii 
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Abstract. - We reformulate the result for the entropy production given in Phys. Rev. Lett. 98, 
080602 (2007) in terms of the relative entropy of microscopic trajectories. By a combination 
with the Crook's theorem, we identify the path variables that are sufficient to fully identify 
irreversibility. We show that work saturates the relative entropy, and derive the entropy production 
for stochastic descriptions. 
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Recent results, known as fluctuation [1-5] or work [6-15] 
theorems, point to the existence of exact equalities that 
rule the fluctuating amounts of work or entropy pro- 
duced during far from equilibrium processes. For exam- 
ple, the Jarzynski equality states that (exp(— (3W)) = 
exp(— (3AF), where W is the work needed to bring a 
system, in contact with a heat bath at temperature T 
(/3 _1 = ksT), from one initial state prepared in equilib- 
rium to another one. AF is the difference in free energy of 
these states (see [10] for a more precise discussion). By the 
application of Jensen's inequality, one finds (W) > AF. 
Since ((W) — AF)/T is the entropy increase in the entire 
construction, system plus heat bath, this result is in agree- 
ment with the second law. While such a result is certainly 
intriguing and of specific interest for the study of small 
systems, where the distribution of work is relevant and 
measurable, it provides no extra information on the ac- 
tual value of the average work or entropy increase, which 
is the central quantity in the second law. Recently how- 
ever, the microscopically exact value of these quantities 
has been obtained in a set-up similar to that of the work 
theorem [16]. The purpose of this letter is to investigate 
some consequences of this result, with special emphasis 
on the case when the dynamics of the system can be de- 
scribed in terms of a reduced set of variables. To make this 
connection, we will rewrite the main result from [16] in an 
alternative form, as an integral over paths. In combina- 
tion with a microscopic version of Crooks' theorem, this 
result identifies the "footprints" of irreversibility, namely 
the path variables whose statistics are sufficient to repro- 



duce the exact total entropy production. This prescription 
is in agreement with the expressions for entropy produc- 
tion proposed in the literature for stochastic models. 

We consider a system described by the Hamiltonian 
H(T, A) with r = ({?}, {p}) a point in phase space, repre- 
senting all position and momentum variables. A is an ex- 
ternal control parameter, for example the volume or an ex- 
ternal field. The system is perturbed away from its initial 
canonical equilibrium at temperature T by changing this 
control parameter according to a specific schedule, from 
an initial to a final value. For simplicity, we will assume 
that during this time the system is disconnected from the 
outside world, except for the action of changing A. This 
assumption makes the derivation and discussion simpler, 
even though the result can be shown to have a much wider 
range of validity [17]. We also consider the time-reversed 
schedule, in which the system starts in canonical equilib- 
rium at the same temperature T, but at the final value of 
the control parameter, and the time-reversed perturbation 
in A is applied. We will use the superscript "tilde" to refer 
to corresponding time-reversed quantities (including, by 
convention, the change of sign for momentum variables). 

The quantity of interest is the amount of work W per- 
formed during the forward process. Since the system is 
isolated, W is equal to the energy difference of the sys- 
tem between final and initial state. While the final state 
is the deterministic outcome, prescribed by Hamiltonian 
dynamics, of the initial condition, the latter is a random 
variable in view of the canonical sampling. Therefore W 
is also a random variable. In the following, it will be use- 
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ful to regard the work If as a functional of the specific 
microscopic trajectory followed by the system. As men- 
tioned above, such a trajectory is completely specified by 
the initial condition, but also by the micro-state T of the 
system at any intermediate time t. In [16], the following 
explicit expression was derived for the corresponding work 
W(T;t): 



W{Y: t)-AF = k B T In 



(i) 



Here p(T;t) and p(f;i) are the phase space densities 
at the same (forward) time t in forward and backward 
experiment. If the system is reconnected after the pertur- 
bation to a (ideal) heat bath at temperature T, the dissi- 
pated work Wdis = W(T; t) — AF will be evacuated to the 
heat bath, resulting in a total entropy production equal to 
Wdis/T. The above formula is thus the microscopic ana- 
logue of the path-dependent entropy production proposed 
in various stochastic models [5, 18-26]. Our emphasis here 
however is on the average dissipated work or average en- 
tropy production. Starting from the same Eq. (1), we 
derive for this average two different expressions, the com- 
bination of which will lead to a general prescription iden- 
tifying the "footprints" of irreversibility. 

First, we derive from Eq. (1) a symmetry relation for 
the probability distribution P(W) of the work as follows: 

P(W) = (S(W -W(T;t))) 

= J dTp(T;t)S(W-W(T-t)) 

= J dre^ w ( r - t )- AF )p(f;t)«5(w-w(r ; t)) 

= e /3(w - AF) y'dfp(f;t)(5(W + W(f;t)) 

= e ^- AF > P(-W), (2) 

since the work in the backward processes verifies 
W(T;t) = — W(T;t). This microscopic Crooks' relation 
was obtained in the context of Markovian stochastic dy- 
namics by Crooks [18], and later extended to Hamiltonian 
dynamics in [14]. The above result is usually viewed as an 
interesting relation for the probability distribution of the 
work. It however also provides a revealing expression for 
the average work. By solving Eq. (2) for W and averaging 
over P(W), one finds: 



(W) 



AF = k B T JdWP(W)\n J 



P(W) 



P(-W) 

= k B T D(P(W)\\P(-W)). (3) 



Here, we introduced the relative entropy, or Kullback- 
Leibler distance, between two probability distributions 
p(x) and q(x) [27]: 



d(p\\q) 



dx p(x) In 



q(x)' 



The relative entropy and its powerful properties will play 
a central role in the sequel. At first sight, it may appear 
superfluous to express the average (W) , which is obviously 
just an integral of P(W), in terms of a more complicated 
expression involving the second probability distribution P 
for the reverse experiment. But the following two impor- 
tant properties of the relative entropy [27] reveal an addi- 
tional benefit. Firstly, a relative entropy is non- negative. 
Eq. (3) thus implies that the dissipated work (W) — AF 
is a positive quantity, in agreement with the second law. 
Secondly, the relative entropy expresses the difficulty for 
distinguishing samplings from two distributions. The dis- 
sipated work is thus equal to the difficulty to distinguish 
the arrow of time from the statistics of the work involved 
in forward versus backward experiment. The main inter- 
est of Eq. (3) however comes from its comparison with an 
expression for (W) in terms of the micro-dynamics, which 
we proceed to derive below. 

By performing the straightforward average in Eq. (1), 
we find [16]: 



(W) - AF = k B T 



ar^toin^ 



p(r ; t) 



k B TD(H|/5). (5) 



In comparison with Eq. (3), the above result fully reveals 
the microscopic nature of the dissipation, but it may ap- 
pear to be of little practical interest. Indeed, it requires 
full statistical information on all the microscopic degrees 
of freedom of the system (even though only at one partic- 
ular time) . This stringent requirement is obviously on par 
with the generality of the above result, which is valid how- 
ever far the system is perturbed away from equilibrium. 
The perturbation could therefore imprint its effect on all 
the degrees of freedom and their complete statistical prop- 
erties would be required to reproduce the corresponding 
dissipation. 

While Eqs. (3) and (5) provide two different exact ex- 
pressions for the dissipated work, we note that the formu- 
las for entropy production in coarse grained descriptions 
are usually in terms of path integrals, on par with the fact 
that the determinism of Hamiltonian dynamics is then re- 
placed by stochastic dynamics. Eq. (3) can be considered 
to be a path integral version since the work W will, in a 
reduced description, indeed depend on the path followed 
by the coarse grained variables during the perturbation. 
To derive a path integral version of Eq. (5), we invoke 
another property of relative entropy [27], known as the 
chain rule. The relative entropy between probability dis- 
tributions p(x, y) and q(x, y) of two random variables can 
be written as follows: 



D(p(x,y)\\q(x,y)) 



+ 



D (p(x)\\q(x)) 

J dxp(x) J dyp(y|x) In 



(6) 

p(yl x ) 
q(y|x) ' 



If the random variables are related to each other by a one- 
^ > to-one function x = f(y), their conditional probabilities 
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become infinitely sharp and the second term in the r.h.s. 
of Eq. (6) vanishes. One then finds: 

D (p(x,y)\\q(x,y)) = D (p(x)\\q(x)) = D (p(y)\\q(y)) . 

(7) 

In words, the addition of dependent variables does not 
modify the relative entropy. Since Hamiltonian dynam- 
ics generates such one-to-one relations between the micro- 
states at different times, one can specify in Eq. (5), with- 
out changing the value of the relative entropy, the micro- 
state r, of the system at as many additional measurement 
points in times ti, i = 1, ...,n, as one likes: 

(W) AF = k B T [ [] dT t p{{T t - U}) m ^ffif" . (8) 

In the continuum limit covering the entire time interval 
(with n — > oo), one thus converges to the following result 
in terms of a path integral, see also [28] : 

(W) — AF — k B T f 2?(path)P(path) In SSS^ 
J P(path) 

= fc B T£>(P(path)||P(path)). (9) 

This expression, while containing redundant information 
from the point of view of Hamiltonian dynamics, has the 
important advantage that it is also exact and formally 
identical, as shown below, when the paths are expressed 
no longer in terms of microscopic variables but in terms 
of an appropriate set of reduced variables. Furthermore, 
in the latter case, the path formulation is no longer re- 
dundant since the trajectory captures information about 
the eliminated degrees of freedom. The identification of 
the minimal set of variables, for which the elimination is 
valid, follows from the combination of Eq. (9) with the 
Crooks' result Eq. (3). One finds: 

D(P(path)||P(path)) = D(P(W)\\P(-W)). (10) 

This is a surprising relationship: from the chain rule for 
the relative entropy, Eq. (6), one would expect that the 
relative entropy for the paths, which contains full informa- 
tion on all the microscopic variables, would be bigger than 
that contained in the work, which is a single scalar path- 
dependent variable. However, both relative entropies are 
exactly the same. The combination of Eqs. (9) and (10) 
allows us now to formulate the following main conclusions. 
First, it is impossible via relative entropy to overestimate 
the dissipation. Second, the exact dissipation is revealed 
by any set of variables that contains the statistical infor- 
mation about the work. The fact that the dissipation is 
underestimated if we do not have this information is also 
of practical interest, but will be the object of a separate 
paper [29]. 

One set of variables that captures the information on 
the work is now easy to identify: the information is ob- 
viously contained in the dynamic variables that are inter- 
acting with the (external) work-performing device. More 



precisely, the work performed along a trajectory T(t), 
t 6 [0, t], is given by: 

and can be exactly calculated from the path followed by 
the variables coupled to A. Then it is enough to know the 
(statistical) behavior of these variables to reproduce the 
statistics of the work, and hence the average dissipation. 
Trajectory information of these and only these variables, 
along the whole (both forward and backward) process, is 
enough to account for the total average dissipation. In par- 
ticular, if a stochastic model provides the exact description 
of a system in its interaction with an external device, one 
needs only the path information of these variables. Eq. (9) 
is thus valid for a "correct" stochastic model with the path 
determined in terms of the corresponding stochastic vari- 
ables. As a corollary, we note that bath variables which 
are replaced (in some ideal limit) by a stochastic pertur- 
bation, will not appear in the "path" , which is in terms of 
the variables of the stochastic system only. 

Let us mention another surprising consequence of the 
above equality (10). By applying the chain rule, Eq. (6), 
one finds: 

D(P(path|W)||P(pa5i|-W))=0 ) (12) 

and hence: 

7>(path|W) = P(path| - W), (13) 

for all W. Eq. (12) means that, by selecting trajectories 
corresponding to a given value of work, W and —W, in 
the forward and backward process respectively, it is not 
possible to detect the arrow of time in them. Accord- 
ing to Eq. (13), the sub-ensembles of these trajectories 
are in fact statistically indistinguishable! As an exam- 
ple, the snapshots of the positions of particles, during the 
expansion and compression of a gas, will be statistically 
identical, when the corresponding amounts of work are 
each other's opposite. For the folding or unfolding of an 
RNA molecule [30], the trajectories are statistically indis- 
tinguishable, again if the amounts of works are opposite. 
We however also note that, according to the Crooks' rela- 
tion Eq. (2), the probabilities for such forward and back- 
ward trajectories will be very different if the experiment 
is performed in an irreversible way. If, e.g., the forward 
set corresponds to typical realizations, the same set of tra- 
jectories will be atypical for the backward experiment [28] 
(except if the overall process is reversible). 

We conclude with a brief discussion about the range of 
applicability of the above result. Recall that, in its deriva- 
tion, it is assumed that the system starts in canonical 
equilibrium in both forward and backward scenario, and is 
disconnected from the heat bath in the intermediate time. 
However, both the microscopic expression for dissipation 
given in Eq. (5) and the Crooks' equality can be derived 
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for other equilibrium initial conditions [17] so that Eq. (9) 
remains valid for these other "transient" noncquilibrium 
scenarios linking equilibrium states. Furthermore, the for- 
mula can also be applied to nonequilibrium steady states 
according to the following argument. Imagine that the 
perturbation induces, after an initial transient, a steady 
state in a sub-part of the system. In the limit that the 
other degrees of freedom for the remainder of the system 
have an infinitely fast relaxation to (local) equilibrium, 
these will not contribute in the formula and both the time- 
irreversibility and dissipation will be completely captured 
by the steady state variables. This hand-waving argument 
explains why the formula (9) is also known to reproduce 
the correct entropy production in nonequilibrium steady 
state models [31,32], where ideal heat, work, and/or parti- 
cle sources are responsible for the generation of the steady 
state. 
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